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ABSTRACT 

Remediation  Technologies  for  Environmental  Projects 

in  the  United  States  Military:  Part  II 

by 

Joseph  Aloysius  Campbell,  M.S. E. 

The  University  of  Texas  at  Austin,  1998 

SUPERVISOR:  James  T.  O'Connor 

This  thesis  analyzes  the  performance  of  environmental  restoration  and 
compliance  projects  in  the  Department  of  Defense.  The  thesis  is  the  second  part 
of  a  two-part  study  examining  project  cost,  schedule,  and  technical  performance. 
The  soundness  of  the  reasons  for  a  specific  remediation  technology  selection  are 
explored  and  tested.  The  research  consists  of  data  collection,  statistical  analysis, 
and  formulating  conclusions  and  recommendations.  This  thesis  demonstrates  that 
planning  environmental  restoration  and  compliance  projects  using  formalized 
decision  matrices  can  increase  the  likelihood  of  project  success. 
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Chapter  1.  Introduction 

1 . 1  Background  and  Motive 

This  thesis  analyzes  the  performance  of  environmental  restoration  and 
compliance  projects  in  the  Department  of  Defense.  The  thesis  is  the  second  part 
of  a  two-part  study  examining  project  cost,  schedule,  and  technical  performance 
on  such  projects.  The  soundness  of  the  reasons  for  a  specific  remediation 
technology  selection  are  explored  and  tested.  The  research  consists  of  data 
collection,  statistical  analysis,  and  the  formulation  of  conclusions  and 
recommendations.  This  thesis  demonstrates  that  planning  environmental 
restoration  and  compliance  projects  using  formalized  decision  matrices  can 
increase  the  likelihood  of  project  success. 

1.2  Purpose  of  this  Research 

The  primary  objective  of  this  research  was  to  formulate  a  better 
understanding  of  the  management  of  environmental  restoration  and  compliance 
projects.  This  study  provides  analysis  of  environmental  remediation  technologies, 
their  performance  from  a  project  manager's  perspective,  and  the  effectiveness  of 
the  reasons  for  their  selection. 

1.3  Research  Scope 

This  research  is  part  of  a  two-part  study  of  project  management  on 
environmental  remediation  projects  in  the  Department  of  Defense  (DOD).  Part  I 
included  the  following  activities: 


•  Research  project  definition 

•  Literature  review 

•  Preparation  of  data  collection  instrument 

•  Data  collection  from  U.S.  Air  Force  sources 

•  Design  and  development  of  relational  database 

•  Recommendations  for  analysis 

This  thesis  is  Part  II  of  the  study  and  includes: 

•  Data  collection  from  U.S.  Navy  sources 

•  Application  of  the  relational  database 

•  Data  analysis  and  presentation  of  conclusions  and  recommendations 

1.4  Research  Hypotheses 

Eight  hypotheses  have  been  tested  in  this  research.  They  include  the  following: 

1 .  That  project  cost  performance  does  vary  with  technology  implemented. 

2.  That  project  schedule  performance  does  vary  with  technology 
implemented. 

3.  That  project  scope  performance  does  vary  with  technology  implemented. 

4.  That  project  scope  performance  does  vary  with  reason  for  technology 
selection. 

5.  That  project  cost  performance  does  not  vary  with  contaminant  type. 

6.  That  project  schedule  performance  does  not  vary  with  contaminant  type. 

7.  That  project  schedule  performance  does  not  vary  with  reason  for 
technology  selection. 


8.    That  project  cost  performance  does  not  vary  with  reason  for  technology 
selection. 

1 . 5  Structure  of  this  Thesis 

Following  this  introductory  chapter,  Chapter  Two  is  dedicated  to 
summarizing  Part  I  of  the  study,  the  work  of  Captain  Scot  T.  Allen,  USAF. 
Chapter  Three  is  a  detailed  explanation  of  the  research  methodology.  Next,  a 
graphical  presentation  of  the  data  and  statistical  data  analysis  is  performed  in 
Chapter  Four.  Chapter  Five  presents  final  conclusions  and  recommendations. 


Chapter  2.  Summary  of  Remediation  Technologies  for 
Environmental  Projects  in  the  United  States  Military:  Part  I 

This  thesis  is  the  second  part  of  a  two-part  study.  This  chapter  is 
dedicated  to  summarizing  Part  I,  the  work  of  Captain  Scot  T.  Allen,  USAF.  A 
complete  copy  of  his  Thesis  is  on  record  at  The  University  of  Texas  at  Austin. 

2.1   Background 

Since  the  enactment  of  the  Comprehensive  Environmental  Response 
Compensation,  and  Liability  Act  (CERCLA)  in  1980,  when  Congress 
established  a  $1.6  billion  "Superfund"  for  environmental  remediation  of  past 
contaminated  sites,  it  has  been  recognized  that  the  costs  of  cleaning  up 
polluted  areas  will  be  several  orders  of  magnitude  higher  than  previous 
estimates  (LaGrega  1994).  Annual  spending  on  environmental  protection  and 
restoration  in  the  U.S.  is  expected  to  reach  $185  billion  by  the  year  2000 
(Kenkeiemath  1996)....  Cost  projections  for  site  remediation  alone  exceed  $1 
trillion  distributed  over  the  next  two  decades  (Blackburn  1993).  (Allen  1997) 

The  U.S.  armed  forces  have  closed  the  era  in  American  history  of 
inattention  to  environmental  issues  and  are  aggressively  pursuing  clean  up 
projects  at  Department  of  Defense  (DOD)  installations.  The  Department  of  the 
Navy  (DON)  has  identified  4433  sites  that  require  environmental  investigation 
and  possible  remediation:   1382  of  the  sites  have  been  remediated,  2549  sites  are 
in  the  study  phase,  and  502  had  a  cleanup  underway  as  of  30  September  1996 
(DON  1996).  The  cost  of  environmental  remediation  is  high.  The  Navy's  Fiscal 
Year  1998  budget  includes  $675  million  for  environmental  studies,  cleanup. 


restoration,  and  compliance.  Base  realignment  and  closure  is  associated  with 
$361  million  of  that  figure  (USA  1998). 

2.2  Management  of  Environmental  Remediation  Projects 

Two  aspects  of  environmental  remediation  project  management  that  differ 
from  conventional  construction  management  are  the  technology  selection  decision 
and  the  way  that  risk  is  managed  in  the  project.  In  the  construction  industry,  risk 
is  assigned  through  legal  contracts  between  owners  and  contractors.  The  most 
common  type  of  construction  contract,  lump  sum  (also  called  firm,  fixed  price) 
assigns  almost  all  of  the  risk  to  the  contractor.  The  cost  reimbursable  contract 
type  assigns  the  majority  of  project  risk  to  the  owner.  The  end  result  of  a 
construction  contract  is  a  tangible  facility  while  the  site  of  a  remediation  project 
may  not  look  significantly  different  to  the  casual  observer  even  after  millions  of 
dollars  have  been  spent.  The  scope  of  an  environmental  remediation  project  may 
be  very  hard  to  distinctly  quantify.  This  increased  uncertainty  in  environmental 
projects  makes  the  contract  type  selection  more  difficult.  According  to  the 
Construction  Industry  Institute  (CII),  "The  unusual  features  of  contaminated  site 
remediation  projects  suggest  that  non-traditional  or  innovative  management  and 
contracting  strategies  may  be  beneficial."  CII  research  indicates  that  contracts, 
which  share  risk,  yield  better  results  with  less  cost  overruns  (CII  1995).  (Allen 
1997) 

2.3  Site  Remediation  Technology 

Captain  Allen  profiled  the  main  remediation  technologies  currently  used  by 
the  U.S.  military.  He  discussed  the  pros  and  cons  of  in  situ  (in  place)  and  ex  situ 
(excavation  /  pumping)  solutions  and  gave  a  good  description  of  the  techniques, 


constraints,  favorable  conditions,  and  cost  estimate  ranges  for  the  following 
technologies: 

Soil  vapor  extraction  Low  permiability  soil  cap 

Air  sparging  Passive  treatment  wall 

Biodegradation  Groundwater  pump  and  treat 

Bioventing  Excavation  and  land  disposal 

Chemical  reduction  /  oxidation  Excavation  and  incineration 

Composting  And  several  innovative  technologies 

Many  remediation  technologies  have  been  developed  to  treat  contaminated 
soil  and  groundwater.  The  Environmental  Protection  Agency  (EPA)  has 
supported  research  on  these  technologies  through  the  Superfund  Innovative 
Technology  Evaluation  (SITE)  program  and  the  Technical  Support  Project 
(Scalf  1992).  Information  on  nearly  350  technologies  is  now  available 
through  the  EPA's  Hazardous  Waste  Clean  Up  Information  Web  site  on  the 
Vendor  Information  Systems  for  Innovative  Treatment  Technologies 
(VISITT)  database.  This  database  can  be  downloaded  for  no  charge  from 
within  the  "Supply  and  Demand"  section  of  EPA's  web  site,  http://clu-in.com 
(EPA  1997).  (Allen  1997) 

2.4       Remediation  Technology  Selection 

Captain  Allen  discussed  three  decision  matrixes  currently  used  by  the  Air 
Force  to  determine  the  optimum  technology  to  address  the  particular  conditions  at 
the  site.  Only  the  two  that  are  used  in  future  analysis  will  be  commented  on  here. 


POD  Treatment  Technologies  Screening  Matrix 

The  Remediation  Technologies  Screening  Matrix  and  Reference  Guide 
provides  a  screening  matrix  for  55  different  remediation  technologies  (see  Table 
2. 1).  These  technologies  have  been  evaluated  based  on  the  following  factors: 
their  development  status  and  commercial  availability,  the  residuals  generated,  the 
contaminants  treated,  reliability  and  maintainability,  schedule,  and  cost.  This 
guide  is  particularly  helpful  to  the  project  manager  faced  with  an  unusual  site  or 
who  wants  to  find  an  appropriate  innovative  technology  (DOD  1994).  (Allen 
1997) 

Air  Force  Center  for  Environmental  Excellence  Remediation  Matrix 

The  Air  Force  Center  for  Environmental  Excellence  (AFCEE)  has 
developed  a  decision-making  tool  entitled  the  Remediation  Matrix-Hierarchy  of 
Preferred  Alternatives  (see  Table  2.2).  This  matrix  provides  a  rank  ordering  of 
remediation  alternatives  for  a  given  contaminant  and  zone  of  contamination  (i.e. 
dissolved  fuel  in  groundwater).  This  remediation  matrix  also  provides  a 
prioritized  list  of  technologies  to  consider  during  project  planning.  Under  a  peer 
review  system  now  in  place  in  the  Air  Force,  remediation  managers  who  elect  not 
to  use  AFCEE' s  recommended  solution  for  a  particular  contamination  scenario 
must  specifically  justify  the  use  of  another  technology  (Allen  1997). 

The  next  section  discusses  the  method  of  study  for  this  research.  The 
sequence  of  analysis,  statistical  analysis,  and  methods  of  handling  data  are 
presented  in  detail. 
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Chapter  3  Study  Methodology 

The  research  procedure  of  the  study  is  shown  in  Figure  3.1.  Part  I  of  this 
study  was  completed  by  Captain  Scot  T.  Allen,  26  August  1997,  and  included 
project  definition,  literature  review,  data  collection  tool  preparation,  database 
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Figure  3.1:  Research  Plan  for  Part  II 
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development  and  collection  of  U.S.  Air  Force  data.  His  research  plan  for  Part  I  is 
included  in  Appendix  A.  This  thesis  included  additional  data  gathering  from  U.S. 
Navy  sources  and  analysis.  As  mentioned  in  Part  I,  "Future  refinement  of  this 
research  could  include  the  collection  of  data  from  the  U.S.  Army,  other 
government  agencies,  or  the  private  sector."  (Allen  1997).  In  this  chapter,  data 
collection  and  analysis  will  be  explained.  Recommendations  for  future  analysis 
will  be  addressed  but  specific  issues  and  additional  data  collection  will  be  fully 
addressed  in  Chapter  5. 

3.1  Development  of  Data  Gathering  Tool  -  Project  Survey 

Captain  Allen  developed  a  data  gathering  tool,  a  "Project  Survey",  with 
input  from  professors  in  the  faculty  of  Construction  Engineering  and  Project 
Management,  Environmental  and  Water  Resources  Engineering,  and 
Geotechnical  Engineering  programs.  His  goals  and  objectives  were  to:   1)  have  a 
short  survey  ?o  that  respondents  would  not  be  dismayed  at  the  task,  and  2)  to 
cover  contamination  type,  geotechnical  conditions,  technology  selected,  reason 
technology  was  selected,  contract  type,  duration  and  cost.  Project  managers  were 
also  asked  to  evaluate  their  projects  considering  cost  and  schedule  performance 
and  numerous  subjective  items.  The  target  time  to  complete  a  survey  was  ten  to 
fifteen  minutes  and  feedback  illustrated  that  this  goal  was  met.  This  survey  in 
included  as  Appendix  B. 

U.S.  Navy  data  collection  for  Part  II  began  in  August  1997.  After 
personally  contacting  project  managers  or  their  supervisors  by  telephone, 
approximately  sixty-nine  data  collection  surveys  were  distributed  to  twenty-three 
project  managers  by  mail,  e-mail,  and  facsimile.  The  data  collection  phase  of  this 
thesis  was  complete  in  mid-November  1997.  Forty-six  of  the  sixty-nine  project 
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surveys  had  been  returned  by  e-mail,  mail,  or  fax  and  were  incorporated  into  the 
database.  Thus  the  response  rate  for  the  second  phase  of  data  collection  was 
approximately  66.7%.  The  willing  participation  of  numerous  engineering  field 
divisions  and  field  activities  far  exceeded  the  goal  of  an  additional  thirty  surveys 
for  Part  II  of  the  study. 

The  combined  data  collection  for  both  Parts  I  and  II  was  very  successful. 
Fifty-three  survey  respondents  provided  data  on  eighty-five  environmental 
remediation  projects.  Twenty-one  of  the  respondents  requested  a  copy  of  the  MS 
Access  ®  database.  Summary  tables  of  the  data  collected  are  included  in 
Appendix  C. 

3.2  Development  of  a  Relational  Database 

Captain  Allen  developed  a  relational  database  with  which  to  store  the  project 
survey  data.  Jn  Part  I  of  this  study,  he  details  the  concepts  and  design  of  the 
relational  database  that  he  developed  using  Microsoft  ®  Access  Office  97  version. 
The  query  and  interface  capabilities  of  the  Microsoft  ®  Office  97  suite  later 
proved  invaluable  in  data  analysis. 

3.3  Part  I  Hypotheses 

In  Part  L,  Captain  Allen  recommended  the  following  hypotheses  be  tested: 

1 .  Projects  in  which  the  guidance  of  the  AFCEE  remediation  technology 
selection  matrix  is  followed  are  more  successful  than  those  which  do  not; 

2.  The  great  majority  (95%)  of  the  technology  selection  decisions  made  in 
military  projects  are  reasonable  based  on  the  site  characterization; 
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3.  Contract  types  which  assign  all  risk  to  the  contractor  or  owner  are  less 
successful  than  risk  sharing  contractual  arrangements;  and 

4.  One  reaches  a  point  of  diminishing  returns  in  site  characterization  and  study, 
beyond  which  project  success  does  not  significantly  improve. 

Part  II  of  the  study  tested  hypothesis  Number  1  above.  This  point  correlates  to 
Part  II  hypotheses  Numbers  7  and  8.  The  remaining  hypotheses  from  Part  I  are 
valid  and  form  the  nucleus  for  recommendations  for  future  analysis,  Section  5.2. 

3.4  Part  II  Research  Scope  and  Objectives 

The  scope  of  Part  II  of  this  research  was  to  gather  data  from  U.S.  Navy  project 
managers  exercising  in  the  field  of  environmental  restoration  and  compliance. 
Once  the  data  was  collected  and  organized  it  was  structured  for  analysis  and 
conclusions  were  made. 

•  Sixty-nine  surveys  sent  to  numerous  Naval  Facilities  Engineering 
Command  Engineering  Field  Divisions  and  Field  Offices  (Part  II) 

•  Thirty-five  Respondents  queried  (Part  II) 

•  Forty-six  Project  Surveys  returned  (Part  II) 

•  Total  Respondents:  Fifty-three  (Parts  I  and  II) 

•  Total  Project  Surveys:  Eighty-five  (Parts  I  and  II) 

The  objectives  of  this  research  were  to: 

•  Collect  data  from  project  managers 

•  Analyze  data 

•  Formulate  conclusions  and  recommendations 

•  Recommend  future  analysis 
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3.5  Data  Collection  from  Project  Managers 

Project  surveys  were  forwarded  to  Navy  environmental  remediation  project 
managers  by  facsimile,  e-mail,  and  mail  after  initial  contact  was  made  by 
telephone.  Resident  Officer  in  Charge  of  Construction  (ROICC)  field  offices, 
Engineering  Field  Divisions  (EFD)  and  the  Navy  Engineering  Service  Center 
were  contacted.  Several  field  offices  referred  to  the  Tulsa  District,  U.S.  Army 
Corps  of  Engineers.  The  Army  Corps  readily  responded  and  provided  three 
project  surveys  from  the  Long  Horn  Army  Ammunition  Plant 

3.6  Database  Development 

The  data  was  entered  into  the  MS®  Access  database  created  in  Part  I  of  the 
study.  The  data  was  queried  and  organized  using  the  query  functions  of  MS® 
Access  and  MS®  Excel. 

3.7  Data  Analysis  Correlations  and  Sequence 

Figure  3.2  shows  the  data  correlations  that  are  analyzed  in  this  thesis.  The 
project  survey  generated  numerous  output  variables  correlated  to  project  inputs. 
Input  variables  are  independent  of  the  process.  In  this  thesis,  environmental 
remediation  project  contaminant,  remediation  technology,  contract  type,  and  site 
geology  are  examples  of  input,  or  independent,  variables.  Output  variables  are 
dependent  upon  the  process  and  one  or  more  input  variables.  Examples  in  this 
study  are  project  cost  and  schedule  performance.  The  first  step  of  analysis  was  to 
consider  the  overall  evaluation  of  projects  based  on  contaminant.  Cost  and 
schedule  performance  was  then  analyzed  versus  contaminant.  A  similar  pattern  of 
analysis  was  followed  for  "Technologies  Selected"  and  "Reason  for  Technology 
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Selection".  Conclusions  were  made  based  on  data  presented,  comparisons 
between  data  sets  and  from  Chi-Square  analysis. 

After  the  most  significant  categories  for  analysis  were  determined,  the  data 
was  presented  graphically  in  the  eleven  variable  relationships  shown  in  Figure 
3.2.  Statistical  analysis  followed.  The  hypotheses  proposed  are  that  relationships 
exist  between  input  and  output  variables.  For  example,  project  cost  and  schedule 
performance  as  well  as  scope  growth  versus  the  type  of  contaminant,  the 
remediation  technology  selected,  and  the  reason  for  technology  selection.  The 
null  hypotheses  tested  are  that  no  such  relationships  exist.  The  chi-square  statistic 
(X2)  was  used  to  test  for  the  existence  of  a  relationship  between  the  variables. 


Input  Variable 

Output  Variable 

•       Contaminant 

^>.                           ""'/f       Overall  success 
/  \    /^-^y        Cost  performance 
/\y             J^      Schedule  performance 

•       Technology  Selected 

•      Reason  Technology  Selected 

e— Scope 

Figure  3.2:  Data  Correlations  Analyzed 

The  chi-square  test  is  a  very  general  test  that  is  used  to  evaluate  whether  or 
not  frequencies  which  have  been  empirically  obtained,  differ  significantly  from 
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those  which  would  be  expected.  Contingency  tables  were  then  constructed 
illustrating  the  cross-classification  of  data.  Table  3.1  is  an  example  of  a 
contingency  table  for  "Cost  vs.  Contaminant".  All  of  the  contingency  tables  are 
included  in  Appendix  D. 


Cost  vs  Contaminant 

Observed  Frequency 

Chlorinated 
Solvents 

Fuel 
Hydrocarbons 

Metals 

Other 
Contaminants 

PCBs 

Row  Total 

Over  Budget                 26.47 

36.59 

20.83 

42.86 

36.36 

163.11 

On  /Under  Budget       73.53 

63.41 

79.17 

57.14 

63.64 

336.89 

Column  Total               100.00 

100.00 

100.00 

100.00 

100.00 

500.00 

Expected  Frequency 

Chlorinated 
Solvents 

Fuel 
Hydrocarbons 

Metals 

Other 
Contaminants 

PCBs 

Row  Total 

Over  Budget                 32.62 

32.62 

32.62 

32.62 

32.62 

163.11 

On  /Under  Budget        67.38 

67.38 

67.38 

67.38 

67.38 

336.89 

Column  Total               100.00 

100.00 

100.00 

100.00 

100.00 

500.00 

Chi-Square  Terms 

Chlorinated 
Solvents 

Fuel 
Hydrocarbons 

Metals 

Other 
Contaminants 

PCBs 

Over  Budget               1.1600 

0.4815 

4.2601 

3.2113 

0.4292 

On /Under  Budget       0.5616 

0.2331 

2.0626 

1.5548 

0.2078 

Chi-Square:                 14.16 

Alpha:                           0.001 

Critical  Value:            18.4662 

Decision:                  Accept  Ho 

Table  3.1  Chi-Square  Contingency  Table 

Step  One:  Compute  the  expected  frequencies  on  the  basis  of  the 
assumption  that  the  variables  are  unrelated  using  the  following  formula: 

fe  =  (  column  total  *  row  total )  /  overall  total 
Where  fe  is  the  expected  frequency. 
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Step  Two:  Compute  the  chi-square  terms  table  using  the  following 
formula: 

X2=(k-fc)2/fe 
Where  f0  is  the  observed  frequency. 

Step  Three:  Determine  the  chi-square  approximation  using  the  following 
formula: 

X2    =    S(  fo  -  fe  )2  /  fe 

Step  Four:  Determine  the  critical  chi  value  from  a  Chi-square  distribution 
table  or  use  the  Microsoft  ®  Excel  CHIINV  function. 

Step  Five:  Compare  the  critical  chi  value  to  the  chi-square  approximation 
determined  in  Step  Three.  Reject  the  null  hypothesis  if  x2  is  greater  than 
the  chi  critical  value.  Do  not  reject  the  null  hypothesis  if  x2  less  than  or 
equal  to  the  chi  critical  value  (Middleton  1995). 

The  null  hypothesis  of  "no  relationship"  implies  that  each  population  will 
have  the  same  proportions  for  each  of  the  categories  of  the  second  variable. 
Looking  at  the  sampling  distribution  of  chi-square  can  test  the  null  hypothesis.    If 
the  value  of  chi-square  is  larger  than  expected  by  chance,  the  null  hypothesis  may 
be  rejected.  The  significance  levels  presented  indicate  the  error  probability  given 
that  the  null  hypothesis  is  rejected.  Thus  smaller  significance  levels  indicated  the 
existence  of  a  possible  relationship  (Blalock  1979). 

3.8  Data  Collection 

The  following  is  a  discussion  about  the  project  survey  development,  data 
gathered  using  the  project  survey,  and  how  it  was  adapted  for  analysis.  The 
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background  on  several  questions  and  the  reasons  for  certain  steps  in  data 
preparations  are  explained. 

Captain  Allen  recommended  in  his  thesis  presentation  that  certain  changes 
be  incorporated  in  the  project  survey.    The  words  "hazardous  waste"  were 
removed  from  the  title  to  avoid  confusion  and  sync  with  the  Environmental 
Protection  Agency's  explicit  definition.  Since  the  word  "failure"  was  considered 
to  be  "too  strong"  in  the  "key  factors"  question  on  the  second  page,  the  survey 
was  changed  to  ask  about  the  "impact  of  key  factors  on  project  outcome  (1- 
positive,  2-no  major  impact,  3-negative,  4-N/A)."  The  final  recommendation  to 
number  the  questions  on  the  survey  was  not  incorporated  and  was  never  an  issue. 
The  discussion  continues  with  background  information  for  one  survey  question. 

The  project  survey  question  "What  is  the  primary  reason  (or  combination 
of  reasons)  for  technology  selection?"  offered  six  selections  for  the  respondent  to 
choose.  The  first  one  was  "Air  Force  Guidance"  which  correlates  to  the  first 
hypothesis  in  Part  I  of  this  study:  "That  projects  in  which  the  guidance  of  the 
AFCEE  remediation  technology  selection  matrix  is  followed  are  more  successful 
than  those  which  do  not."  Part  II  of  the  study  focused  on  data  collection  from  the 
U.S.  Navy  so  the  response  selection  on  the  survey  was  changed  to  read 
"NAVFAC  Guidance."  NAVFAC  does  not  utilize  the  Air  Force  decision 
selection  matrix  so  the  response  to  this  question  could  not  be  combined  into  an 
overall  category  such  as  "Sponsor  Guidance."  NAVFAC  has  a  general  policy  that 
innovative  technologies  should  be  utilized  in  an  effort  to  optimize  schedule  and 
cost  performance  but  does  not  adhere  to  a  strict  decision  matrix  (DON  1996). 
Thus,  this  study  will  address  the  use  of  the  AFCEE  remediation  technology 
decision  matrix  and  its  effectiveness  as  a  primary  reason  for  technology  selection. 
The  next  two  sections  will  describe  data  that  fell  into  the  "Other"  categories. 
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The  "Other  Technology"  category  has  twenty-six  projects  in  it.  Five  of 
the  projects  used  a  combination  of  technologies  generally  associated  with  land 
disposal.  This  category  of  "Other  Technologies"  includes  many  innovative  means 
for  environmental  clean  up  and  compliance,  including:  underground  storage  tank 
and  fuel  piping  removal,  resin  adsorption  vapor  treatment,  base  catalyzed 
decomposition,  and  recycling  various  material  for  asphalt  concrete. 

The  "Other  Contaminants"  category  is  associated  with  fifteen  projects. 
Seven  projects  have  one  type  of  contamination  not  falling  into  one  of  the  four 
major  categories  while  eight  have  a  combination  of  contaminants.  Generally,  the 
"Other  Contaminants"  are:  pesticides,  low  level  radioactive  waste,  lead,  asbestos, 
and  explosives. 

Data  was  arranged  for  graphical  presentation  increasing  from  left  to  right 
with  the  "good"  category  in  the  series  to  the  rear.  The  author  did  this  since  the 
"good"  category  was  almost  always  numerically  greater  than  the  "bad"  category 
and  the  graph  was  therefore  easier  to  read.  The  "good"  category  is  typically 
series  such  as  "On  Budget",  "Ahead  or  On  Schedule",  or  "Successful".    The  next 
two  sections  discuss  data  not  utilized  due  to  insufficient  sample  size  and  grouping 
data  into  larger  categories. 

Some  data  could  not  be  analyzed  due  to  insufficient  response  in  that 
particular  category.  Respondents  were  given  full  latitude  to  select  the  projects 
that  they  reported  on  although  they  were  requested  to  try  to  give  a  quality  spread 
with  one  project  considered  highly  successful,  one  project  considered  typical,  and 
one  project  below  their  expectations.  Several  of  the  remediation  technology 
categories  did  not  have  enough  data  to  be  statistically  valid. 
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Technologies  dropped  include: 

Passive  Treatment  Wall  Bioventing 

Chemical  Oxidation  /  Reduction  Incineration 

Much  of  the  data  gathered  on  the  Project  Survey  was  suitable  for  grouping 
prior  to  analysis.  The  data  was  grouped  for  several  reasons.  First,  there  was  a 
very  fine  line  in  distinction  between  alternatives  for  respondents  to  select  on  some 
of  the  subjective  questions.  And  second,  grouping  simplified  and  clarified 
analysis.  Three  categories  of  data  were  grouped  during  this  analysis:   1) 
"Successful"  and  "Acceptable"  were  grouped  in  the  project's  overall  assessment . 
This  was  grouped  because  the  survey  did  not  present  sufficient  ranking  criteria  or 
structure  to  differentiate  the  two.  Additionally,  "Acceptable"  implies  a  success. . . 
the  two  are  very  nearly  the  same.  2)  "Ahead"  and  "On  schedule"  were  grouped 
because  "On  schedule"  is  good  and  "Ahead  of  schedule"  is  generally  accepted  as 
good  also.  3)  "No  change"  and  "Decreased  scope"  were  grouped  because  "No 
change"  in  scope  suggests  that  the  scope  definition  was  good  and  "Decreased 
scope"  is  generally  accepted  as  good. 

The  data  collection  tool,  while  itself  concise,  generated  far  more  areas  of 
study  than  can  be  adequately  addressed  in  one  thesis  presentation.  The  data 
collected  are  included  in  Appendix  C.    The  names  and  telephone  numbers  of  the 
individual  survey  participants  are  not  provided  in  this  thesis  for  confidentiality. 
The  next  two  sections  propose  future  analysis.  The  first  section  centers  on  a 
wealth  of  subjective  data  on  factors  impacting  project  outcome.  The  respondent 
was  requested  to  rank  fourteen  items  one  to  four  using  the  following  scale: 
1  -  Positive  2  -  No  major  impact 

3  -  Negative  4  -  Not  applicable 
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The  second  section  is  a  good  follow-on  to  analysis  in  this  thesis  considering 
contract  issues,  geotechnical  issues,  and  clean-up  standards  versus  overall 
success,  cost  and  schedule  performance,  and  scope  change. 

3.9  Recommended  Future  Analysis 

Data  items  not  considered  in  this  analysis  but  reserved  for  possible  future 
analyses  include  the  following: 


Output  Variables 

Operations  and  maintenance  costs 

Percentage  complete  (to  date) 


Input  Variables 
Sources  of  funding 
Estimated  contract  cost 
Project  duration 
Reuse  plans  for  the  site 
Impact  on  project  outcome 

Project  planning  /  Funding 

Political  involvement 

Laboratory  analysis   /  Sampling  plan  &  methods 

Implementation  contract  type  /  Contractor  performance 

Team  building  &  partnering  /  Contract  disputes 

Severe  weather  /  Contract  incentives 

Discovered  more  contamination 

Unanticipated  soil,  geological  or  ground  water  conditions 

Technology  performance 


Data  analyses  considered  particularly  valuable  but  reserved  for  future 
analyses  are  shown  in  Figure  3.3.  The  entire  spectrum  of  contract  type  is  useful 
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Input  Variable 

Output  Variable 
c- ~Zsfl       Overall  success 

vsA  Y//^     Cost  performance 
//[\/\\---~-~S        Schedule  performance 

•       Contract  Type 

•       Groundwater  affected 

•       Applicable  clean-up  standards 

Technology  vs.  Geology 

•       Site  Characterization  Study  Cost 

f- Scope 

Figure  3.3:  Correlations  Recommended  for  Future  Analyses 

to  consider  when  faced  with  an  environmental  compliance  requirement.  Ground 
water  and  site  geology  analysis  could  yield  indicators  of  success  in  one 
remediation  technology  over  another.  The  clean-up  standards  that  are  applied  by 
various  agencies  vary,  can  affect  project  success,  and  are  certainly  worthy  of 
analysis.  The  evolution  of  site  characterization  costs  is  valuable  to  consider  for 
many  reasons.  One  could  determine  whether  there  are  trends  over  the  past  fifteen 
to  twenty  years  showing  that  pre-project  planning  is  paying  off  and  in  what 
particular  arena.  Perhaps  more  importantly,  one  may  determine  if  the  Department 
of  Defense  is  getting  better  at  dealing  with  environmental  issues. 

The  next  chapter  covers  data  analysis.  Again,  Figure  3.2  illustrates  the 
sequence  of  analyses  and  is  a  ready  reference  guide  through  Chapter  4. 
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Chapter  4.  Data  Analysis 

4.1  Project  Performance  versus  Contaminant 

The  vast  majority  of  the  projects,  97.7%  were  evaluated  as  successful.  As 
shown  in  Figure  4.1,  only  two  of  eighty-five  projects  in  the  sample  were  rated 
unsuccessful. 
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Successful  Projects 
Unsuccessful  Projects 


Figure  4. 1 :  Overall  Performance  vs.  Contaminant  Type 

The  overwhelming  survey  response  with  successful  projects  may  in  part  be 
due  to  respondents'  natural  tendency  to  report  on  successes  rather  than  failures 
even  though  a  quality  spread  was  requested.  Table  4.1  details  contaminant  type  in 
the  unsuccessful  projects.  Project  "A"  had  fuel  hydrocarbon  contamination  and 
Project  "B"  had  a  combination  of  fuel  hydrocarbon,  chlorinated  solvents,  and 
metals.  Both  projects  were  over-budget,  behind  schedule,  and  their  scope 
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increased.  The  final  question  on  the  project  survey  was  an  overall  evaluation  of 
the  project  results  to  date.  The  respondent  was  given  three  categories  from  which 
to  select  a  response: 

•  Successful 

•  Acceptable 

•  Unsuccessful 

As  discussed  in  Section  3.7,  "Successful"  and  "Acceptable"  were  grouped 
together  to  clarify  analysis.  In  retrospect,  providing  a  better  metric  for  response 
to  this  question  would  have  significantly  increased  the  value  of  the  data.  The 
following  two  sections  show  that  while,  for  the  data  gathered,  fuel  hydrocarbons 
were  present  in  a  significant  number  of  projects  associated  with  poor  cost  and 
schedule  performance,  statistical  analysis  shows  in  both  cases  that  there  is  no 
relationship  between  cost  or  schedule  performance  and  contaminant  type. 


Contaminant  Present 

Unsuccessful 
Project  "A" 

Unsuccessful 
Project  "B" 

Other  Contaminants  (n=14) 

Fuel  Hydrocarbons  (n=41) 

X 

X 

PCBs(n=11) 

Chlorinated  Solvents  (n=34) 

X 

Metals  (n=24) 

X 

Table  4. 1 :  Contaminants  Present  in  Unsuccessful  Projects 
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4.2  Cost  Performance  versus  Contaminant 

The  data  shown  in  Figure  4.2  suggest  that  there  are  few  budget  certainties 
in  environmental  restoration  and  compliance  projects. 


On  or  Under  Budget 
Over  Budget 


Figure  4.2:  Cost  Performance  vs.  Contaminant  Type 

Overall  twenty-nine  of  the  eighty-five,  or  34.1%  of  the  projects  sampled  were 
over  budget.  Projects  with  metal  contaminants  performed  best.  Both  "Fuel 
Hydrocarbons"  and  "Other  Contaminants"  categories  performed  lower  than 
average  for  the  sample  set  when  considering  cost  performance.  While  graphically 
it  appears  that  projects  with  metal  contaminants  perform  better  than  other 
contaminants,  chi-square  statistical  analysis  shows  that  project  cost  performance 
in  general  does  not  vary  with  contaminant  type.  The  contingency  tables  that 
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support  the  chi-square  analysis  of  the  null  hypothesis  are  included  in  Appendix  D, 
Table  D-l. 

Specific  observations  include  the  following: 

•  Metals  -  Five  of  twenty-four  or  20.8%  over  budget 

-  Associated  data  show  that  scope  increased  on  four  of  the  five  projects  over 
budget. 

•  Fuel  Hydrocarbons  -  Fifteen  of  forty-one,  or  36.6%  over  budget 

-  Scope  increased  on  seventeen  of  the  forty-one  projects.  Not  all  of  the 
budget  overruns  are  possibly  attributed  to  scope  growth  though  as  only 
twelve  of  those  were  over  budget  (29.3%  of  the  total). 

•  Other  Contaminants    -  Six  of  fourteen  or  42.9%  over  budget 

-  Scope  increased  on  nine  of  fourteen  projects,  six  of  which  were  over 
budget  (42.9%  of  the  total). 

-  As  discussed  in  Section  3.8,  contaminants  in  this  category  include 
pesticides,  low  level  radioactive  waste,  lead,  asbestos,  and  explosives 
which  coupled  with  scope  growth  may  explain  the  over-budget  cost 
performance. 

4.3  Schedule  Performance  versus  Contaminant 

The  data  shown  in  Figure  4.3  suggests  that  four  of  the  five  categories 
performed  satisfactorily  "Ahead  or  On  Schedule"  73%  to  82%  of  the  time. 
Overall  sixty-three  of  the  eighty-five,  or  74. 1  %  of  the  projects  sampled  were 
"Ahead  or  On  Schedule"  schedule.  "Chlorinated  Solvents"  were  the  best 
performers  while  both  the  "Fuel  Hydrocarbons"  and  the  "Other  Contaminants" 
categories  performed  lower  than  average  for  the  sample  set.  While  graphically  it 
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appears  that  projects  with  chlorinated  solvent  contaminants  perform  better 
regarding  schedule  than  other  contaminants,  chi-square  statistical  analysis  shows 
that  project  schedule  performance  in  general  does  not  vary  with  contaminant  type. 
The  contingency  tables  that  support  the  chi-square  analysis  of  the  null  hypothesis 
are  included  in  Appendix  D,  Table  D-2. 
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Figure  4.3:  Schedule  Performance  vs.  Contaminant  Type 

Specific  observations  include  the  following: 

•     Chlorinated  Solvents  -  Six  of  thirty-four  or  17.7%  behind  schedule. 
-    Four  of  these  six  had  scope  growth. 
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•  PCBs  -  Two  of  eleven  or  1 8.2%  behind  schedule. 

•  Metals  -  Six  of  twenty-four  or  25%  behind  schedule. 

•  Fuel  Hydrocarbons  -  Eleven  of  forty-one  or  26.7%  behind  schedule. 

•  Other  Contaminants  -  Six  of  fourteen  or  42.9%  behind  schedule. 

-  Scope  increased  on  nine  projects,  six  of  which  were  over  budget  (42.9%  of 
the  total). 

-  As  discussed  in  Section  3.8,  contaminant  types  in  this  category  include 
pesticides,  low  level  radioactive  waste,  lead,  asbestos,  and  explosives 
which  coupled  with  scope  growth  may  explain  this  over  budget  cost 
performance. 

4.4  Project  Performance  versus  Technology  Implemented 

The  project  survey  identified  eleven  specific  environmental  remediation 
technologies  and  allowed  the  respondent  to  pencil  in  any  additional  innovative 
technologies  that  may  have  been  utilized: 

Soil  vapor  extraction  Passive  treatment  wall 

Air  sparging  Low  permiability  soil  cap 

Biodegradation  Goundwater  pump  and  treat 

Bioventing  Excavation  and  incineration 

Chemical  reduction  /  oxidation  Excavation  and  land  disposal 

Composting  And  several  innovative  technologies 

Bioventing,  composting,  treatment  wall,  incineration,  and  chemical  oxidation  / 
reduction  are  not  considered  in  this  study  due  to  insufficient  sample  population  in 
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the  response  to  the  survey.  Project  cost  and  schedule  performance  as  well  as 
scope  change  and  overall  evaluation  will  be  addressed  in  this  section. 

4.5  Overall  Project  Evaluation  versus  Technology  Implemented 

All  but  two  of  the  eighty-five  projects  surveyed  were  judged  by  the  survey 
respondents  to  be  successful. 
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Figure  4.4:  Overall  Project  Results  vs.  Technology  Implemented 

One  soil  vapor  extraction  project  and  one  pump-and-treat  project  were 
over  budget,  behind  schedule,  and  increased  in  scope  more  than  5%.  Their 
project  manager  judged  both  projects  unsuccessful.  As  discussed  in  Section  4. 1, 
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clearer  definition  of  a  more  refined  metric  would  have  produced  more  valuable 
data  in  this  category.  Figure  4.4  displays  this  data  graphically. 

4.6  Cost  Performance  versus  Technology  Implemented 

Figure  4.5  illustrates  that  seven  of  eight  remediation  technologies 
performed  well  being  on  or  under  budget  between  66.7%  and  100%  of  the  time. 
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Figure  4.5:  Cost  Performance  vs.  Technology  Implemented 

"Composting"  performed  poorly  with  all  four  of  the  projects  in  the  sample 
group  over  budget.  While  four  projects  in  each  category  is  a  small  sample  size, 
two  technologies  stand  out  in  their  cost  performance.  "Air  Sparging"  performed 
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particularly  well  with  dissolved  fuel  or  dissolved  chlorinated  solvents  in  ground 
water  and  "Composting"  performed  poorly  from  the  sample  set  with  fuel 
contaminated  soil.  Statistical  analysis  supports  the  graphical  representation  in 
Figure  4.5  that  suggests  that  some  remediation  technologies  are  better  than  others 
in  cost  performance.  Chi-square  analysis  rejects  the  null  hypothesis  and  supports 
the  alternate  hypothesis  that  project  cost  performance  does  vary  with  technology 
implemented.  The  contingency  tables  that  support  the  chi-square  analysis  of  the 
null  hypothesis  are  included  in  Appendix  D,  Table  D-3. 

Specific  observations  include  the  following: 

•  "Soil  Vapor  Extraction"  -  Two  of  twelve  or  13.7%  of  the  projects  were  over 
budget. 

•  The  best  performer  was  "Air  Sparging"  with  100%  of  the  four  projects  on  or 
under  budget. 

•  All  four  projects  utilizing  "Composting"  were  over  budget. 

-  Associated  data  shows  that  three  of  the  four  projects  were  also  behind 
schedule  and  three  of  the  four  (not  the  same  three)  increased  in  scope 
more  than  5%. 

•  "Low  Permeabilitiy  Soil  Cap"  -  Two  of  the  six,  or  33.3%  of  the  projects  were 
over  budget. 

•  "Other  Technologies"  -  Four  of  thirteen  or  30.8%  of  the  projects  were  over 
budget. 

-  As  described  in  Section  3.8,  often  developing  or  innovative  technologies 
were  in  this  category.  Five  such  projects  which  were  a  combination  of 
technologies  generally  associated  with  land  disposal  and  underground 
storage  tank  and  fuel  piping  removal,  resin  adsorption  vapor  treatment, 
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base  catalyzed  decomposition,  and  recycling  various  material  for  asphalt 
concrete  were  over  budget. 

4.7  Schedule  Performance  versus  Technology  Implemented 

The  data  in  Figure  4.6  illustrates  that  five  of  the  eight  remediation 
technologies  performed  well  being  ahead  or  on  schedule  between  76.9%  and 
100%  of  the  time. 
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Figure  4.6:  Schedule  Performance  vs.  Technology  Implemented 

Land  disposal  and  biodegradation  were  the  top  performers  while  soil  vapor 
extraction,  composting,  and  air  sparging  were  the  poorest.  Statistical  analysis 
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supports  the  graphical  representation  in  Figure  4.6  that  suggests  that  some 
remediation  technologies  are  better  than  others  in  schedule  performance.  Chi- 
square  analysis  rejects  the  null  hypothesis  and  supports  the  alternate  hypothesis 
that  project  schedule  performance  does  vary  with  technology  implemented.  The 
contingency  tables  that  support  the  chi-square  analysis  of  the  null  hypothesis  are 
included  in  Appendix  D,  Table  D-4. 

Specific  observations  include  the  following: 

•  Nine  of  the  ten  "Land  Disposal"  and  five  of  five  "Biodegradation"  projects 
were  ahead  or  on  schedule. 

•  "Composting"  -  Three  of  the  four,  or  75%,  of  the  projects  were  behind 
schedule. 

•  "Air  Sparging"  -  Two  of  the  four,  or  50%,  of  the  projects  were  behind 
schedule. 

•  "Soil  Vapor  Extraction"  -  Five  of  the  twelve,  or  41.7%,  of  the  projects  were 
behind  schedule. 

Schedule  and  cost  performance  on  the  four  projects  utilizing  "Composting"  may 
be  tied  together.  All  four  projects  were  over  budget  and  three  of  the  four  were 
behind  schedule. 

4.8  Scope  Change  versus  Technology  Implemented 

The  data  in  Figure  4.7  illustrates  that  project  scope  increased  between  30% 
and  50%  in  the  best  performing  six  remediation  technologies  implemented.  The 
series  displayed  in  Figure  4.7  is  bracketed  by  two  technologies  that  had  a  much 
smaller  sample  size  of  four  projects  each.  While  four  projects  in  each  category  is 
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a  small  sample  size,  two  technologies  stand  out  in  their  change  in  scope.    "Air 
Sparging"  performed  particularly  well  with  dissolved  fuel  or  dissolved 
chlorinated  solvents  in  ground  water  and  "Composting"  performed  poorly  from 
the  sample  set  with  fuel  contaminated  soil.  Statistical  analysis  supports  the 
graphical  representation  in  Figure  4.7  that  suggests  that  some  remediation 
technologies  are  better  than  others  in  scope  change.  Chi-square  analysis  rejects 
the  null  hypothesis  and  supports  the  alternate  hypothesis  that  project  scope  change 
does  vary  with  technology  implemented.  The  contingency  tables  that  support  the 
chi-square  analysis  of  the  null  hypothesis  are  included  in  Appendix  D,  Table  D-5. 
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Figure  4.7:  Scope  Change  vs.  Technology  Implemented 

Specific  observations  include  the  following: 

•  "Air  Sparging"  -  One  of  four,  or  25%  of  the  projects  increased  in  scope. 

•  "Land  Disposal"  -  Three  often,  or  30%  of  the  projects  increased  in  scope. 


35 


•  "Pump  and  Treat"  -  Six  of  thirteen,  or  46.2%  of  the  projects  increased  in 
scope. 

•  "Soil  Vapor  Extraction"  -  Six  of  twelve,  or  50%  of  the  projects  increased  in 
scope. 

•  "Composting"  -  Three  of  four,  or  75%  of  the  projects  increased  in  scope. 

4.9  Performance  versus  Reason  for  Technology  Selection 

This  section  presents  an  analysis  of  project  performance  in  cost,  schedule, 
scope  change,  and  overall  success  versus  the  reason  the  program  manager  selected 
the  particular  remediation  technology.  The  survey  addressed  six  major  reasons 
why  program  managers  selected  a  specific  remediation  technology: 

Selection  may  be  based  on  guidance  from  AFCEE  or  NAVFAC. 

"Regulatory  requirements"  may  dictate  a  specific  technique. 

"Minimal  Exposure  Hazard"  may  be  a  concern. 

"Cost"minimization. 

"Schedule"  maintainability  or  quick  turn-around. 

"Effectiveness"  of  the  technology. 
Section  3.8  covers  in  detail  what  is  meant  by  AFCEE  or  NAVFAC  guidance.  The 
Air  Force  has  developed  a  decision  matrix  for  remediation  technology  selection. 
Part  of  this  study  is  to  validate  that  matrix. 

4.10  Overall  Project  Evaluation  versus  Reason  for  Technology  Selection 

The  project  survey  queried  the  respondent's  "Evaluation  of  overall  project 
results  to  date"  and  allowed  the  respondent  to  select  one  of  three  categories:    a) 
Successful,  b)  Acceptable,  or  c)  Unsuccessful. 


36 


Ptrcantag* 
of  Projects 


Successful 

Unsuccessful 


Figure  4.8:  Overall  Results  vs.  Reason  for  Technology  Selection 

Figure  4.8  illustrates  overall  project  results  versus  each  of  the  six 
categories  that  survey  respondents  selected  as  the  primary  reason  or  combination 
of  reasons  for  technology  selection.  All  six  categories  were  graded  exceptionally 
well.  Overall  there  were  only  two  of  eighty- five  projects  or  2.4%  marked 
unsuccessful.  Each  of  these  projects  was  behind  schedule,  over  budget,  and  the 
scope  had  increased. 

The  data  supports  the  first  hypothesis  from  Part  I  of  this  study,  but  overall 
the  results  may  be  inflated.  Grade  inflation  may  be  a  combination  of  an 
insufficient  scale  on  the  project  survey  attempting  to  quantify  this  subjective  data 
and  the  survey  respondents  desire  to  inflate  overall  project  evaluation.  It  is  only 
natural  to  want  to  point  out  one's  successes  rather  than  one's  lesser  performance. 
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When  schedule,  cost,  and  scope  are  considered  with  the  overall  project  evaluation, 
from  a  combined  perspective,  the  data  supports:  "that  projects  which  utilize  the 
AFCEE  remediation  technology  selection  matrix  were  more  successful  than  those 
which  do  not".  As  discussed  in  Sections  4. 1  and  4.5,  clearer  definition  of  a  more 
refined  metric  would  have  produced  more  valuable  data  in  this  category. 

4.11  Cost  Performance  versus  Reason  for  Technology  Selection 

The  project  survey  queried  project  cost  and  allowed  the  respondent  to 
select  one  of  three  categories:  a)  Under  budget  (2%  or  more),  b)  On  budget,  or  c) 
Over  budget  (2%  or  more).  Figure  4.9  illustrates  cost  performance  versus  each  of 
the  six  categories  that  survey  respondents  selected  as  the  primary  reason  or 
combination  of  reasons  for  technology  selection.  Generally,  five  categories 
performed  in  an  acceptable  range  being  on  budget  between  67.6%  and  75%  of 
the  projects  studied.  When  "Schedule"  was  selected  as  a  reason  for  the  project's 
technology  selection,  the  overall  results  were  below  average.  Graphically  it 
appears  that  projects,  which  used  AFCEE  guidance  or  effectiveness  to  select  the 
remediation  technology  to  be  implemented,  performed  best  regarding  cost.    Chi- 
square  statistical  analysis  shows  that  project  cost  performance  in  general  does  not 
vary  with  reason  for  technology  selection.  The  contingency  tables  that  support 
the  chi-square  analysis  of  the  null  hypothesis  are  included  in  Appendix  D,  Table 
D-6. 
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Figure  4.9:  Cost  Performance  vs.  Reason  for  Technology  Selection 

Specific  observations  include  the  following: 

•  When  "Schedule"  was  selected  as  a  reason  for  the  project's  technology 
selection,  projects  were  only  on  budget  10  of  17  or  58.8%  of  the  time. 

-  Project  duration  or  schedule  may  have  been  more  important  than  cost  and 
the  project  manager  may  have  allowed  the  cost  to  creep  up  in  order  to 
maintain  schedule. 

•  When  "Cost"  was  a  technology  selection  criteria,  projects  came  in  on  budget 
67.6%  of  the  time. 

-  Twenty-one  of  the  thirty-eight  projects  addressed  fuel  hydrocarbon  or  a 
combination  of  fuel  and  other  contaminants.  Eight  of  twelve,  or  66.7%,  of 
the  over  budget  projects  were  to  remediate  fuel  contamination. 

-  Four  of  the  twelve,  or  33.3%  of  the  over  budget  projects  were  to  remediate 
chlorinated  solvents. 
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-  "Composting"  and  "Land  Disposal"  were  the  predominant  remediation 
technologies  in  these  projects. 

•  When  "Regulatory  Requirements"  was  a  technology  selection  criteria, 
projects  came  in  on  budget  69.2%  of  the  time. 

•  When  "Minimal  Exposure  Hazard"  was  a  technology  selection  criteria, 
projects  came  in  on  budget  72.3%  of  the  time. 

•  "Effectiveness"  of  the  technology  is  probably  the  strongest  selection  reason 
with  72.9%  of  the  large,  48  project,  sample  being  on  budget. 

-  This  reason  was  most  often  cited  as  the  reason  that  a  project  manager 
selected  a  particular  technology.  "Effectiveness"  was  selected  56.5%  of 
the  time. 

-  When  the  technology  is  selected  based  on  how  well  it  performs,  tried  and 
true  methods  deliver  in  the  majority  of  the  cases  sampled. 

•  Projects  that  utilized  the  AFCEE  technology  selection  matrix  in  determining 
which  remediation  technology  to  utilize  were  successfully  on  budget  75%  of 
the  time. 

Using  the  AFCEE  selection  matrix  as  the  primary  reason  for  technology 
selection  was  the  best  overall  selection  criterion  for  this  data  sample.  This  area  of 
analysis  confirms  part  of  the  first  hypothesis  from  Part  I. 

4.12  Schedule  Performance  versus  Reason  for  Technology  Selection 

The  project  survey  queried  project  schedule  performance  and  allowed  the 
respondent  to  select  one  of  three  categories:  a)  Ahead  of  schedule  (2%  or  more), 
b)  On  schedule,  or  c)  Behind  schedule  (2%  or  more). 
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Figure  4. 10:  Schedule  Performance  vs.  Reason  for  Technology  Selection 

The  data  illustrated  in  Figure  4.10  shows  schedule  performance  versus 
each  of  the  six  categories  that  survey  respondents  selected  as  the  primary  reason 
or  combination  of  reasons  for  technology  selection.  Generally,  all  six  categories 
performed  in  an  acceptable  range  being  on  schedule  between  73.7%  and  90.9%  of 
the  projects  studied.  Graphically  it  appears  that  projects,  which  used  minimal 
exposure  hazard  or  AFCEE  guidance  to  select  the  remediation  technology  to  be 
implemented,  performed  best  regarding  schedule.  Chi-square  statistical  analysis 
shows  that  project  schedule  performance  in  general  does  not  vary  with  reason  for 
technology  selection.  The  contingency  tables  that  support  the  chi-square  analysis 
of  the  null  hypothesis  are  included  in  Appendix  D,  Table  D-7. 
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Specific  observations  include  the  following: 

•  The  most  outstanding  categories  in  this  sample  were  "Minimal  Exposure 
Hazard"  and  "AFCEE  Guidance"  which  were  ahead  or  on  schedule  90.9% 
and  87.5%  for  the  eleven  and  eight  projects  respectfully.    When  the  program 
manager  selected  a  remediation  technology  to  address  one  of  these  criteria,  he 
was  typically  dealing  with  the  following  type  projects: 

-  Six  of  the  eight  projects  that  utilized  the  "AFCEE  Guidance"  had  fuel 
hydrocarbon  or  a  combination  of  fuel  and  other  contaminants. 

-  The  Air  Force  projects  did  not  utilize  "Composting"  but  used  "Soil  Vapor 
Extraction",  "Biodegradation",  "Bioventing",  and  in  one  case  "Land 
Disposal". 

-  Six  of  the  eleven  projects,  or  55%,  that  utilized  the  "Minimal  Exposure 
Hazard"  "  as  a  remediation  technology  (and  used  "Cost"  as  a  technology 
selection  criterion)  addressed  a  chlorinated  solvent  contamination.  Four  of 
these  six,  or  36%  of  the  projects  were  for  metal  contamination.  Land 
disposal  or  soil  cap  were  used  almost  exclusively. 

•  Projects  that  selected  a  technology  specifically  for  "Schedule"  concerns  came 
in  ahead  or  on  schedule  fourteen  of  seventeen  or  82.3%  of  the  time  ranking 
third  overall. 

•     The  low  end  of  the  spectrum  were  projects  with  technology  selected  based  on 
"Cost"  which  still  produced  a  satisfactory  twenty-eight  of  thirty-eight  or 
73.7%  ahead  or  on  schedule.     When  the  program  manager  selected  a 
technology  based  on  cost  control,  there  are  several  reasons  that  relate  to  poor 
schedule  performance: 

-  Twenty-one  of  the  thirty-eight  projects  that  were  behind  had  fuel 
hydrocarbons  or  a  combination  of  fuel  and  other  contaminants.  Six  of 
these  twenty-one,  or  29%  were  behind  schedule. 
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-  Four  of  the  ten  projects  that  were  behind  schedule  were  utilizing 
innovative  technologies.  Two  of  these  projects  were  using  processes  that 
are  preliminary  to  "Composting". 

-  Three  of  four  projects,  or  75%,  that  utilized  "Composting"  as  a 
remediation  technology  (and  used  "Cost"  as  a  technology  selection 
criterion)  were  behind  schedule. 

-  Three  of  seven  projects,  or  40%,  that  utilized  "Soil  Vapor  Extraction"  as  a 
remediation  technology  (and  used  "Cost"  as  a  technology  selection 
criterion)  were  behind  schedule. 

This  data  suggests  that  "Composting"  does  not  have  as  effective  cost  control 
as  other  remediation  technologies  and  is  a  slower  process  than  originally 
programmed  thus  additional  time  is  required  during  project  execution. 
"Composting"  should  not  be  selected  as  a  remediation  technology  if  schedule  and 
cost  are  important.  This  data  is  also  associated  with  analysis  in  Section  4.7 
showing  "Soil  Vapor  Extraction"  projects  to  be  behind  schedule  on  five  of  twelve, 
or  41 .7%  of  the  time.  This  correlation  suggests  that  while  "Soil  Vapor 
Extraction"  is  a  good  performer  for  cost  control,  it  is  not  good  in  schedule 
performance. 

Using  the  AFCEE  selection  matrix  as  the  primary  reason  for  technology 
selection  was  a  strong  performer  for  this  data  sample.  This  area  of  analysis 
confirms  part  of  the  first  hypothesis  from  Part  I. 

4.13  Scope  Change  based  on  Reason  for  Technology  Selection 

The  project  survey  queried  project  scope  change  and  allowed  the 
respondent  to  select  one  of  three  categories:  a)  Increased  scope  (5%  or  more),  b) 
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No  change,  or  c)  Decreased  scope  (5%  or  more).  The  data  in  Figure  4. 1 1 
illustrates  scope  change  versus  each  of  the  six  categories  that  survey  respondents 
selected  as  the  primary  reason  or  combination  of  reasons  for  technology  selection. 
Scope  change  is  common  place  in  environmental  remediation  projects  due  to  the 
inherent  uncertainty  associated  with  unknown  and  underground  conditions.  A 
project  whose  scope  increases  is  not  necessarily  considered  unsuccessful.  In  fact, 
the  scope  increased  on  fifty-two  of  eighty-five  or  61.2%  of  the  projects  surveyed 
yet  only  two  projects  were  reported  unsuccessful.  Statistical  analysis  supports  the 
graphical  representation  in  Figure  4. 1 1  suggesting  that  some  reasons  for  selecting 
a  remediation  technology  are  better  than  others  in  scope  definition.  Chi-square 
analysis  rejects  the  null  hypothesis  and  supports  the  alternate  hypothesis  that 
project  scope  performance  does  vary  with  reason  for  technology  selection.  The 
contingency  tables  that  support  the  chi-square  analysis  of  the  null  hypothesis  are 
included  in  Appendix  D,  Table  D-8. 

Specific  observations  include  the  following: 

•  Both  of  the  unsuccessful  projects  were  over  budget,  behind  schedule,  and  the 
scope  increased. 

•  Scope  control  as  an  output  of  the  reason  for  technology  selection  is  best 
afforded  when  "Schedule"  or  "Effectiveness"  of  technology  is  most  important. 

-  "Effectiveness"  -  Three  of  forty-eight,  or  6.25%  increased  scope. 

-  "Schedule"  -  Three  of  seventeen,  or  17.7%  increased  scope. 

•  "AFCEE  selection  criterion"  -  two  of  eight,  or  25%  increased  scope. 

•  "Cost"  -  Nineteen  of  thirty-eight  projects,  or  50%  increased  in  scope. 
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Figure  4.11:  Scope  Change  vs.  Reason  for  Technology  Selection 

This  may  not  be  cause  for  alarm  when  correlated  with  other  survey  data. 
None  of  the  projects  marked  "unsuccessful"  by  survey  respondents  used  cost  as  a 
primary  reason  for  technology  selection.  Only  nine  of  the  nineteen  projects  that 
experienced  scope  growth  were  over  budget.  In  this  category  there  are  thirty- 
eight  projects  which  considered  "Cost"  as  a  primary  reason  for  technology 
selection.  Nine  projects,  or  23.7%  experienced  scope  growth  coupled  with  being 
over  budget.  This  cost  performance  coupled  with  scope  increases  is  considered  to 
be  within  satisfactory  bounds.  When  the  overall  project  rating  and  budget 
concerns  are  considered  in  concert  with  the  scope  change,  the  data  suggests  that 
project  controls  were  successful  when  "Cost"  was  considered  important.  The  next 
chapter  presents  the  study  conclusions  and  recommendations. 
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Chapter  5  Conclusions  and  Recommendations 

5.1  Conclusions 

This  chapter  completes  the  study  by  summarizing  the  hypotheses  and  then 
presentation  of  conclusions  and  recommendations.  The  following  hypotheses 
were  proven  in  this  thesis: 

1 .  That  project  cost  performance  does  vary  with  technology  implemented. 

2.  That  project  schedule  performance  does  vary  with  technology 
implemented. 

3.  That  project  scope  performance  does  vary  with  technology  implemented. 

4.  That  project  scope  performance  does  vary  with  reason  for  technology 

selection. 

This  thesis  demonstrates  that  steps  in  planning  environmental  restoration  and 
compliance  projects  can  increase  the  likelihood  of  successful  project  performance. 
Specifically,  careful  consideration  of  the  reason  for  technology  selection  and  the 
actual  technology  selected  can  greatly  effect  project  outcome  in  schedule  and  cost 
performance.  Both  the  Department  of  Defense  Technology  Selection  Matrix  and 
the  Air  Force  Center  for  Environmental  Excellence  Remediation  Matrix  are 
effective  decision  making  tools  to  help  select  the  appropriate  remediation 
technology  although  they  do  not  in  themselves  guarantee  success.  These  tools  are 
certainly  recommended  for  the  inexperienced  project  manager.  Additional 
hypotheses  also  proven  in  this  thesis  follow: 

5.  That  project  cost  performance  does  not  vary  with  contaminant  type. 

6.  That  project  schedule  performance  does  not  vary  with  contaminant  type 
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7.  Project  schedule  performance  does  not  vary  with  reason  for  technology 

selection. 

8.  Project  cost  performance  does  not  vary  with  reason  for  technology 

selection. 

The  following  conclusions  are  in  addition  to  the  proven  hypotheses: 

1.  That  in  general  the  AFCEE  selection  matrix  is  a  valuable  tool  in 
determining  which  remediation  technology  to  utilize. 

2.  That  while  the  Air  Force  pushes  composting  in  the  AFCEE  matrix  it  was  a 
poor  performer  in  this  sample  group. 

5.2  Recommendations 

The  following  recommendations  were  developed  during  analysis  and 
evaluation  in  this  thesis: 

1 .  That  further  study  be  conducted  using  the  abundant,  valuable  data  already 
gathered  on  the  project  survey  and  formulated  in  the  database.  The  most 
valuable  data  relationships  to  consider  are  shown  in  Figure  3.3.  Contract 
type,  groundwater  problems,  applicable  clean-up  standards,  technology 
compared  to  geology,  and  site  characterization  study  costs  can  all  be 
compared  to  overall  project  success,  project  cost  and  schedule 
performance,  and  project  scope  changes. 

2.  That  additional  data  should  be  gathered  to  specifically  address  the 
question  of  DOD  performance  in  site  characterization.  A  study  of  the 
evolution  of  site  characterization  costs  could  determine  whether  there  are 
trends  over  the  past  fifteen  to  twenty  years  showing  that  pre-project 
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planning  is  paying  off  and  in  what  particular  arena.  More  importantly,  one 
may  determine  if  the  Department  of  Defense  is  getting  better  at  dealing 
with  environmental  issues  and  also  whether  the  costs  of  the  study  produce 
sufficient  benefit  to  justify  continued  expenditures. 

3.  That  should  additional  data  be  gathered  using  this  project  survey  tool  or 
one  similar  to  it,  the  question  of  an  overall  project  evaluation  should  be 
refined  and  a  better  metric  should  be  developed.  The  metric  could  build 
on  a  scale  of  one  to  five  and  give  quantifiable  items  for  the  respondent  to 
consider.  Successful  projects  would  be  graded  five  and  unsuccessful 
projects  graded  one.  Acceptable  projects  would  be  defined  as  a  grade  of 
three.  The  respondent  would  be  asked  to  subtract  one  point  if  the  project 
was  behind  schedule  2%  or  more  and  subtract  one  point  if  the  project  was 
over  budget  2%  or  more.  Other  similar  quantifiable  items  could  be 
defined  or  the  balance  of  the  grade  could  be  left  to  the  respondent's 
subjective  evaluation.  The  end  result  would  be  much  more  valuable  data 
for  project  performance  analysis. 

4.  That  a  systems  engineering  approach  should  be  used  in  the  remediation 
technology  selection  process.  That  is,  decisions  should  be  made  using 
some  sort  of  proven  decision-making  matrix  such  as  the  Department  of 
Defense  Remediation  Technologies  Screening  Guide,  Table  2.1  or  the  Air 
Force  Center  for  Environmental  Excellence  Remediation  Matrix- 
Hierarchy  of  Preferred  Alternatives,  Table  2.2.  Both  tables  have  been 
developed  with  process  improvement  and  feedback  loops  to  self-improve. 

This  concludes  the  written  portion  of  this  thesis.  Following  are  various 
appendices  including  data  and  analysis  tables,  then  bibliography,  and  vita. 
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Appendix  A 
Research  Plan  from  Part  I 


The  following  figure  is  the  work  of  Captain  Scot  T.  Allen  USAF. 
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Appendix  B 
Data  Collection  Tool 


The  following  survey  is  the  work  of  Captain  Scot  T.  Allen  USAF. 
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Please  fill  out  and  return  to:    LT  Joseph  A.  CampbeU  Tel:  (512)  331-8899 

Department  of  Civil  Engineering,  CEPM      E-mail: 
jacampbeU  amail.  utexas.edu 

The  University  of  Texas  Fax:  (512)471-3191 

Austin  TX  78712-1076 


Name 


Environmental  Site  Remediation  Project  Survey 

Fax:  Date: 


Agency/Unit: 
Telephone:  _ 
E-mail:  


Project  Name: 


Project  Location  (Base,  City,  State): 


The  Problem: 


Contaminants  present  (check  all  that  apply): 
Q    Fuel  hydrocarbons 
Q    Chlorinated  solvents 
Q    Metals 
Q    PCB's 
Q    Other 


Maximum  depth  of  contamination: 

Q 

0-10  feet 

□ 

11-20  feet 

a 

21-30  feet 

a 

31-40  feet 

a 

41-50  feet 

Q 

Over  50  feet 

Contamination  has  affected  (all  that  applv) 

a 

Soil 

a 

Groundwater 

a 

Air 

If  groundwater  is  affected,  contaminants  are  (check  all 

that  apply): 

Q    Dissolved  in  groundwater 
□    Free  product  (Non-Aqueous  Phase 
Uquid.  NAPL) 


If  groundwater  is  affected,  the  plume: 
Q    extends  beyond  the  property  line 
Q    is  completely  on  site 

□  hat  an  unknown  extern 

Average  depth  to  the  water  table: 

Q  0-10  feet 

□  11-20  feet 
O  21-30  feet 
Q  31-40  feet 
Q  41-50  feet 

□  Over  50  feet 

Soil/geology  classification  (check  the  most  important  sin 
features): 

□  Tight  clay/silt  (impermeable  soils) 
G    Loose  sand/gravel  (permeable  soils) 
Q     Relatively  impermeable  bedrock  (e.g. 

solid  granite) 

□  Permeable  bedrock  (eg.  fissured 
limestone) 

Site  is  planned  for  reuse: 

□  In  1-3  years 
Q    In  4-10  years 

□  No  definite  plans  (or  no  information  i 


Remediation  technology  selected  (please  indicate 

combinations  i : 

Q  Soil  vapor  extraction  (SVE) 

Q  Air  sparging  {in  situ) 

Q  Biodegradatjon  (except  biovenung ) 

Q  Biovenung 

□  Chemical  Oxidation/Reduction 
Q  Composting  or  Land  Farming 
Q  Excavation  and  land  disposal 
Q  Excavation  and  incineration 

□  Low  Permeability  Soil  Cap 

□  Passive  Treatment  Wall 

□  Pump  and  treat  (ex  situ  air  stripping; 

□  Other 


The  Solution: 


Applicable  clean-up  standards: 

□  Non-detect  level 
O    Background  level 

□  Risk  based  clean-up  level 

□  Federal  or  state  remediation  standard 

Primary  reasons  this  technology  (or  combination  i 
was  selected: 

Q     NA  VFAC  guidance 

□  Cost 

□  Schedule 

□  Regulatory  requirements 

□  Effectiveness 

Q    Minimal  exposure  hazard 

Q    Other  
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Soured  s)  of  funding: 

3     Base  realign  mem  and  closure  (BRAG 
Q     Installation  ResioraQon  Program  (IRP) 


Q     Defense  Environmental  Restoration 

Account  (DERA) 
Q    Other 


The  Contract: 


Type  of  remediation  implementation  contract: 
Q     Finn  fixed -pnee  (lump  sum) 
Q    Cost  reimbursable  (cost  plus) 
O     Unit  pnee 
Q    Other  


Estimated  total  contract  cost  amount  ( investigation, 
implementation,  momtonng): 


Implementanon  contract  project  duration 
(months): 


Design  and  construction  were  done  by: 
Q     Separate  contracts 

□  In-faouse  design  and  separate 
construction  contract 

Q    Design-build  contract 

What  percentage  of  the  implementation  project  lias 
been  completed  to  date: 

O    0-25% 

Q    26-50% 

□  51-75% 
Q     76-100% 

Q     Project  complete 


Results  to  Date: 


Project  scope  change  during  project: 
Q    Increased  (5%  or  more) 
Q    No  change 

□  Reduced  (5%  or  more) 

Project  cost: 

□  Under  budget  (2%  or  more) 

□  On  btiQpet 

Q    Over  budget  (2%  or  more) 

Anticipated  or  actual  annual  operations  and 
maintenance  (O&M)  cost:  


Percentage  of  total  project  cost  spent  on  site 
charaaenzanon  and  study  


Schedule  performance: 

G    Ahead  of  schedule  ( 2%  or  more  > 

□  On  schedule 

Q    Behind  schedule  (2%  or  more) 

Project  met  (or  is  meeting)  regulatory 
remediation  goals: 

□  Yes 
Q    No 


Impact  on  project  outcome  (1 -Positive. 
2-No  major  impact.  3-Negatrve,  4-N/A): 


12    3    4 

Project  planning 

G  G  G  G 

Sampling  plan/methods 

q  a  a  a 

Laboratory  analysis 

a  a  a  a 

Lmplementanon  contract  type 

q  a  a  a 

Contract  incentives 

a  a  a  a 

Contract  penalties 

a  a  a  a 

Team  budding/partnering 

a  a  a  a 

Contractor  performance 

□  a  a  a 

Contract  disputes 

a  a  a  a 

Discovered  more  contaminahon 

a  a  a  a 

Unanucipaied  soil  geological. 

QQQQ 

or  groundwater  conditions 

Technology  performance 

QQQQ 

Severe  weather  {force  majeure) 

QQOQ 

Funding 

QQQQ 

Political  involvement 

QQQQ 

Other 

QQQQ 

Evaluation  of  overall  project  results  to  date: 

O    Successful 

G    Acceptable 

G    Unsuccessful 

Would  you  like  a  disk  copy  of  the  Microsoft  Access  for  Windows  database? 
Other  comments  on  the  project  (or  any  of  the  questions  above): 


□  Yes 


QNo 


Please  recommend  another  person  who  could  contribute  to  this  research  by  filling  out  project  information 

surveys: 

Name:  E-mail:  Tel: 

Address  Fax: 
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Appendix  C 
Data  Tables 

Table  C-l  Project  Name  and  Location  -  Data  Table 56 

Table  C-2  Project  Contaminant  and  Comments  -  Data  Table 59 

Table  C-3  Project  Reason  for  Technology  Selection  and  Applicable  Clean-up 

Standards  -  Data  Table 64 

Table  C-4  Project  Results  -  Data  Table 68 

Table  C-5  Contract  Type  and  Cost  -  Data  Table 72 

Table  C-6  Contract  Funding  and  Performance  -  Data  Table 76 

Table  C-7  Extent  of  Contamination  -  Data  Table 80 

Table  C-8  Regulatory  Goals  and  Remediation  Technology 83 

Table  C-9  Site  Geology 87 
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At  conclusion  of  remediation  contract,  no  further  action 
required,  so  there  is  no  O&M  cost. 

Treatment  system  is  operational  and  is  controlling  plume. 
O&M  to  continue  until  levels  of  TCE  are  below  5  ppb. 

Landfill  wastes  must  be  excavated  to  eliminate  contact  with 
groundwater  and  capped. 

Scope:  remove  52  -  50,000  gal  underground  storage  tanks 
(USTs)  and  10  -  2,000  gal  USTs  and  contaminated  soil. 
Remove  all  distributed  piping  not  paved  over. 

Site  characterization  with  goal  of  natural  attenuation.  Former 
hydrant  system  within  100  feet  of  public  water  well. 

Need  to  contain  plume  and  remediate  site  quickly  as  it  is 
located  in  a  National  Register  of  Historic  Places  site 
Newington  Town  Forest.  Pump  and  treatd  for  containment 
only.  75%  of  the  surface  was  capped. 

Originally  scoped  as  SVE.  Changed  to  excavation  because 
of  supposed  high  water  table.  Project  is  much  more 
expensive  initially.  No  O&M  costs  following  excavation. 
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Operation  on  the  treatment  system  will  not  be  continuous  if 
the  contaminant  is  treated  on  a  consistent  level  below  the 
MCL. 
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This  project  is  a  new  technology  demonstration.  Material 
recycled  in  asphalt  mix. 
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Team  building/partnering  was  the  most  effective  part  of  the 
work  in  reducing  cost  and  saving  time. 

Select  monitored  natural  attenuation  with  hot  spot 
remediation  of  soil  on  a  40  acre  landfill  with  residual  soil  and 
GW  contamination  saved  about  $4M  in  construction  and  over 
$10M  in  O&M  costs 
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Appendix  D 
Chi-Square  Contingency  Tables 

Table  D-l  Chi-Square  Contingency  Table  for  Cost  vs.  Contaminant 
Table  D-2  Chi-Square  Contingency  Table  for  Schedule  vs.  Contaminant 
Table  D-3  Chi-Square  Contingency  Table  for  Cost  vs.  Technology 
Table  D-4  Chi-Square  Contingency  Table  for  Schedule  vs.  Technology 
Table  D-5  Chi-Square  Contingency  Table  for  Scope  Change  vs.  Technology 
Table  D-6  Chi-Square  Contingency  Table  for  Cost  vs.  Reason  for  Technology 

Selection 
Table  D-7  Chi-Square  Contingency  Table  for  Schedule  vs.  Reason  for 

Technology  Selection 

Table  D-8  Chi-Square  Contingency  Table  for  Scope  Change  vs.  Reason  for 
Technology  Selection 
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